Least kth-Order and Rényi Generative Adversarial Networks

Abstract We investigate the use of parameterized families information-theoretic measures to generalize loss functions generative adversarial networks (GANs) with objective improving performance. A new generator function, least kth-order GAN (LkGAN), is introduced, generalizing squares GANs (LSGANs) by using a absolute error distortion measure k≥1 (which recovers LSGAN function when k=2). It shown that minimizing this generalized under an (unconstrained) optimal discriminator equivalent Pearson-Vajda divergence. Another novel next proposed in terms Rényi cross-entropy functionals order α>0, α≠1. demonstrated Rényi-centric which provably reduces original as α→1, preserves equilibrium point satisfied based on Jensen-Rényi divergence, natural extension Jensen-Shannon Experimental results indicate functions, applied MNIST and CelebA data sets, both DCGAN StyleGAN architectures, confer performance benefits virtue extra degrees freedom provided parameters k α, respectively. More specifically, experiments show improvements regard quality generated images measured Fréchet inception distance score training stability. While it was study, approach generic can be used other applications information theory deep learning, for example, issues fairness or privacy artificial intelligence.